Anomaly Detection in Networks with Changing Trends

Dynamic networks, also called network streams, are an important data representation that applies to many real-world domains. Many sets of network data such as e-mail networks, social networks, or internet traffic networks have been analyzed in the past mostly using static network models and are better represented by a dynamic network due to the temporal component of the data. One important application in the domain of dynamic network analysis is anomaly detection. Here the task is to identify points in time where the network exhibits behavior radically different from a typical time, either due to some event (like the failure of machines in a computer network) or a shift in the network properties. This problem is made more difficult by the fluid nature of what is considered ”normal” network behavior: a network can change over the course of a month or even vary based on the hour of the day without being considered unusual. A number of anomaly detection techniques exists for standard time series data that exhibit these kinds of trends but adapting these algorithms for a network domain requires additional considerations. In particular many different kinds of network statistics such as degree distribution or clustering coefficient are dependent on the total network degree. Existing dynamic network anomaly detection algorithms do not consider network trends or dependencies of network statistics with degree and as such are biased towards flagging sparser or denser time steps depending on the network statistics used. In this paper we will introduce a new anomaly detection algorithm dTrend which overcomes these problems in two ways: by first applying detrending techniques in a network domain and then by using a new set of network statistics designed to be less dependent on network degree. By combining these two approaches into the dTrend algorithm we create a network anomaly detector which can find anomalies regardless of degree changes. When compared to current techniques, dTrend produces up to a 2x improvement in F1 score on networks with underlying trends. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ODD’14, August 24th, 2014, New York, NY, USA. Copyright 2014 ACM 978-1-4503-2998-9 ...$15.00.